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In this research, we aim to understand the effectiveness of adopting educational technologies in a 
computer literacy course to students in a medical university. The course was organized with three core 
components: Open Education Resources (OER) reading, a book club, and online game competition. 
These components were delivered by a learning management system (LMS). Participating records of 
LMS and survey results are analyzed. This study has shown positive results in terms of students’ self - 
evaluation and online participation rate. 
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INTRODUCTION 

The professional society for educational technology, the 
Association for Educational Communications and 
Technology (AECT), had defined: “educational technology 
as the study and ethical practice of facilitating learning 
and improving performance by creating, using, and 
managing appropriate technological processes and 
resources.” It emphasizes on integrating diverse media to 
improve teaching and learning. Most educational techno¬ 
logies are implemented through information technologies. 
Learning management systems (LMSs) may become one 
of the most common means in providing effective 
teaching and learning (Junco and Clem, 2015). 

Using LMS, students can gain more access to class 
materials and video lectures instructors deployed. In LMS, 


synchronous online discussions provide valuable profiles 
for evaluating different pedagogical interventions 
(Kovanovic et al., 2015).However, both students and 
faculty need help to make the best practice of educational 
technology (Abelson et al., 2011). To provide these 
assistance, various educational strategies have been 
examined, such as, shared mental model (Sikorski et al., 
2011), collaborative group modeling (Lee et al., 2014), 
meta-analytic evaluation (Adedokun et al.,2014), peer 
instruction (Vickrey et al., 2015), blended learning 
(Graham et al., 2013), virtual intercultural interaction 
(O'Dowd, 2013), and role assignment and participation 
(Xie et al., 2014). Through such techniques, educational 
technology can improve students' performance and 
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instructor-independent. However, the study also shows 
that the student participation rates were extremely 
variable (Wolter et al., 2012). In many researches, 
educational technologies were pedagogical tools 
developed for teachers. As the self-regulated learning 
becomes a relevant and valuable concept in higher 
education (Cassidy, 2011), the implementation of 
educational technologies need to consider further on 
various learning styles of students. In this research, 
through collecting both objective and subjective data for 
the past six years from a computer literacy course using 
LMS. We would like to demonstrate the effectiveness of 
innovative course activities design, such as using OERs, 
book club, and game in an online course. 

The concept of OpenCourseWare (OCW) was initiated 
by Massachusetts Institute of Technology (MIT) in 2000 
(Yue and Chen, 2004; Yue and W., 2004). Beginning in 
2002, Hewlett Foundation founded various developments 
of open educational resources to improve quality of 
teaching and learning. MIT’s OCW was one of the funded 
programs (Atkins et al., 2007). By 2006, the OCW 
Consortium included more than 100 universities. MIT had 
published 1800 courses by fall 2007 (Abelson, 2008a). 
Most OCWs consist of teaching materials, sample 
assignments and quizzes, offered weekly. Recently, 
UNESCO announced open education resource (OER) in 
its Paris declaration. The OERs include the publication of 
open courses that anyone may access on the Internet 
and re-use (DeVries, 2013). Due to the international 
promotion of OCW and OER, projects such as teacher 
education in sub Saharan Africa (TESSA) program 
(Murphy and Wolfenden, 2013) have been reported to 
drive pedagogical changes. In China, Chinese Quality 
Course (CQC) project was launched in 2003 to promote 
the awareness of OER (Hu, Li, Li, and Huang, 2015). 
With the growth of OERs, search tools or platforms have 
been developed. OER repositories had been built as well 
(Marcus-Quinn and Diggins, 2013). Studies such as 
metadata(Tang et al., 2013) and folk semantic (Shelton et 
al., 2010) have been reported. 

To monitor the quality of online programs and students’ 
performance, students’ satisfaction surveys have been 
used (Kuo et al., 2014). Many aspects of online learning 
self-efficiency has been evaluated, such as interaction 
between classmates and instructors, and online platform 
handling (Shen et al., 2013). During the past decade, 
information literacy has been listed as a core competence 
in health science education. American Library Association 
(ALA) published the ‘Information Literacy Competency 
Standards for Higher Education’ in 2000. It states that 
‘Information literacy’ is common to all disciplines, to all 
learning environments, and to all levels of education. 
‘Information literacy’ enables learners to master content, 
to extend their investigations, to become more self- 
directed, and assume greater control over their own 
learning. Information technology skills relate to information 


literacy. At present, students should be able to learn skills 
of critical thinking and problem-solving, collaboration 
across networks, and accessing and analyzing 
information (Nielsenet al., 2014).To achieve a wide 
variety of academic, work-related, and personal goals, an 
individual needs training in information technology skills. 
In many higher education organizations, information 
literacy or computer literacy courses had been added into 
the curriculum. A computer literacy course for life 
sciences may include modules such as, web-based 
project repositories, databases, background literature 
research, data analysis, and presentation (Smolinski, 
2010). For students with life science majors, information 
literacy courses consisting of basic computer concept, 
library usage, and applications in healthcare may be 
designed into a curriculum. In this research, we would 
like to know if a computer literacy course is effective to 
students in a medical university with adopting educational 
technologies. Both quantitative and qualitative analyses 
of students’ performance and feedback were preformed 
and will be presented. The analytics of LMS data will be 
one of the means to present the effectiveness of course 
activities. Moreover, we would also like to know if student 
majors affect their learning outcomes. If any, what would 
be the possible cause? 


MATERIAL AND METHODS 
Course sample 

The basic computer concept (BCC) course is an online computer 
literacy course. The course is implemented as an entry-level 
election for undergraduate in a medical university. Topics include 
introduction to hardware, software, Internet, information security 
and privacy, copyright and creative commons, and related 
applications. Students are expecting to spend 2 h efforts each week 
for 17 weeks to finish the course. The objectives of this introductory 
course are (1) how to use information technology (IT) tools to 
present their ideas, (2) where to find answers or resources of their 
homework or research through Internet or databases, and (3) how 
to cooperate with each other through internet. 

The BCC course is delivered through the university's LMS. The 
LMS provides functions for delivering all course activities. Topics 
and instructions are released on a weekly basis. Students are 
required to give their feedback through discussion forums or 
surveys. There are three innovative activities provided in this course: 

1. Self-learning from Open Education Resources (OER). 

2. Cooperative learning from a Book club. 

3. Game-learning from an online Jeopardy-like game. 

The course has been offered for several years. Enrollments are 
also open to students of various colleges or schools from time to 
time. Approximately 60 students enroll in each academic year. 
Objective data such as user reading clicks, reading time and 
discussion rate are collected directly from system, while subjective 
data are gathered through student surveys, writing assignment and 
peer reviews. By 2013, the BCC course had been offered for 7 
consecutive years. A total of 441 students had enrolled, with 90% of 
them being non-native speakers of English. Only records of 374 
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students taking the course after the LMS change in 2008 were 
studied in this research. 


Using OER 

An OER, ‘Computer Science-El: Understanding computers and the 
internet’ (El), is the primary knowledge source for the course 
studied in this research. This OER was used at Harvard University 
Extension School, and was first published in fall, 2005. It is a video 
courseware filmed in the classroom. The first version was filmed at 
Harvard University and consisted of 14 lectures with David Malan 
as the lecturer. David is one of the pioneers in filming OERs of 
computer science. A newer version with 9 lectures was filmed in 
spring 2011. David’s apprentice, Dan Armendariz, and David Malan 
were the lecturers. The courseware is under a Creative Commons 
Attribution-Non commercial-Share Alike 3.0 Unsupported License. 
The original web site of El lists course highlight in one page, which 
is easier for self-learning. To integrate the OER, we add more 
selected topics in the LMS and arranged links to the videos. After 
examining the course videos, students are required to submit the 
weekly assignments. Progress of student self-learning is monitored 
automatically by the LMS. 


Book club 

The book for this activity is “The Pattern on the Stone", by W. 
Daniel Hiliis, and consists of 9 chapters. Students are divided into 9 
groups. Each group is assigned to read one chapter of the book, 
respectively. Each group after the reading makes a video 
presentation and shares the learning with other groups. During the 
semester, 3 presentations each week for 3 weeks are allotted for 
these group activities. All students give evaluation and feedback on 
each presentation as part of the course requirement. 


Online Jeopardy-like game 

A Jeopardy game, featuring categories and graduated answer 
values, is a popular quiz competition. The online competition in the 
BCC course consists of a 3 week activity. In the preparation phase, 
each team submits a category and 5 quizzes related to topics learnt 
from the course. Each quiz is labeled a value from 100 to 500.The 
game is a role-based competition. There are three roles in this 
competition, the chairman, the secretary general, and the 
spokesperson. When the spokesperson post questions, the 
secretary generals of other team could answer these posted 
questions. Then, the chairman has to check the correctness of 
answers and determine who win the score. Competition teams with 
4 to 6 students are formed and each team chooses persons to 
assume the 3 roles respectively. During the competition, each team 
shall follow the defined instructions to find the best strategy of 
obtaining the highest score and win. All of these activities are 
accomplished by using functions provided by the LMS. 


Data collection 

The standardized end-of-semester learning survey consists of 15 
scale questionnaires related to course satisfaction plus 1 open 
question. Another survey at the end of semester focusing on e- 
learning experiences presents 20 scale questionnaires and 1 open 
question. Peer reviews are collected at the end of the book club 
and online competition, respectively. There are two kinds of peer 


reviews, in-team and between-team. Between team, peer review is 
done after the book club presentation, each student uses a rubric 
rate in the presentation team. The in-team peer review is for team 
members to evaluate each other. The total point of each student 
earned is calculated as a portion of his or her final grade. All these 
data are collected by LMS. Moreover, in the LMS logs every click of 
a user on a content page (a view) as well as the time between one 
click to another defines the ‘reading time’. 


Outcome assessment 

Three primary outcomes were collected and assessed: (1) the 
engagement of learners with the activities, (2) participation of online 
activities, and (3) grading from various assignments. Quantitative 
and qualitative analyses of surveys provided the engagement data. 
Content analyses of the open questions were used for qualitative 
evaluation. Participation data came from LMS logs. Quantitative 
descriptive statistical analyses were related to the participation and 
grades. Moreover, in this research, the major of a student was used 
as one of the variables, and performance differences among majors 
were analyzed. 


RESULTS 

Engagement with the activities 

A total of 371 student surveys were used in this research. 
Overall, 65% of the students claimed that they had read 
more than 75% of OER content. There were 30% of 
students who considered the Harvard content as the 
most difficult learning activity. Only 7.5% of students liked 
the English speaking OERs. The major reason was the 
language barrier. Some students suggested video 
subtitles or lecture notes which would help learning. 
Nearly 35% of students agreed that courseware from 
prestigious universities was motivating their learning. 
Only 9.5% of students thought online competition was 
impressive. Merely 15.6% of students considered book 
club interesting. General speaking, 46% of students liked 
this course to be an e-learning course. Time-flexibility 
and activity-diversity were the two major benefits from 
this course. More than a half of students considered 
participation of course activities as “virtual teams” was 
hard since many needed to call for help on keeping their 
teammates in touch. 


Participation of online activities 

Views and discussions generated by students and 
collected by the LMS were counted for activity 
participation. From 2008 to 2013, a total of 139,629 views 
and 426 teaching materials were recorded. Views per 
weeks per year statistics are shown in Figure 1. Total 
views per week are presented in Figure 2. The first peak 
appears between week 4 and 8 when the first team 
activity starts. The second one is between week 10 and 
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Figure 1. Student views in weeks of each year’s course. 
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Figure 2. Total student views in each week. 


12, when the book club activity begins. The third is 
around week 17 when the online competition kicks in. 
According to the LMS logs, the video, how to make a 
presentation video, apparently was the most viewed 
teaching material. Following “the instruction of the book 
clubs” came second. Among the 4,130 discussions, there 
was a mean of 11 discussions per student with a median 
of 8 which was recorded. A medical student hit the 
highest frequency of 62 discussions. 

Grading from various assignments 

Both in-team and between-team peer reviews of team 


presentations represented a portion of the team 
presentation score. Final grade of a student came from 
homework, team presentation, competition scores, online 
activities, as well as peer reviews. The average of student 
grades was 82.2% with a median of 87.4%. A positive 
correlation between a student’s final grade and his/her 
course reading records was demonstrated, as shown in 
Figure 3. Students who spent more time on the learning 
content generally scored higher in grades. 

Differences among majors 

The BCC course is a general election course for all 
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Figure 3. Content views versus final grades. 


Table 1. Average grades of students in different majors. 


Majors 

n 

Drop-out 

Average 

SD 

Max 

Medicine 

82 

0 

83.16 

18.04 

97.85 

Dental 

46 

0 

85.36 

15.69 

97.50 

Pharmacy 

59 

1 

86.06 

8.90 

98.10 

Med Tech 

27 

0 

78.79 

14.23 

96.00 

Nursing 

14 

0 

76.57 

21.27 

94.10 

Nutrition 

53 

0 

84.51 

12.19 

97.10 

Public Health 

25 

0 

72.24 

24.31 

97.10 

Health Administration 

9 

0 

83.08 

11.65 

96.40 

Respiratory Therapy 

11 

1 

79.38 

10.04 

98.04 

Gerontology Health Mgt 

15 

0 

82.42 

16.33 

98.19 

Dental Tech. 

22 

0 

83.80 

11.68 

96.20 

Allied Universities 

11 

1 

68.31 

20.53 

93.85 


students in a medical university. Enrolled students vary in 
their majors and in academic years. In this survey, 22% 
of enrolled students are from school of medicine and 
15.8% from school of pharmacy. Students’ performances 
in their majors are shown in Tables 1 and 2. Comparing 
the average grades, students in pharmacy appears to 
score slightly higher than all other medical students. 
Comparing the records of views and discussions, 
students in public health and nursing tend to view less 


and talk less. 


DISCUSSION 

Many interesting videos can be introduced to non¬ 
computer science major students to facilitate their 
computer literacy. A course designed both for those with 
little, if any, computer experience and for those who use 
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Table 2. Average participation data of students in different majors. 


Majors 

n 

Discussion 

View 

V/D 

Medicine 

82 

11.4 

111.4 

9.8 

Dental 

46 

13.1 

119.1 

9.1 

Pharmacy 

59 

11.5 

116.3 

10.1 

Med Tech 

27 

10.0 

102.3 

10.3 

Nursing 

14 

8.7 

97.1 

11.1 

Nutrition 

53 

9.9 

104.0 

10.5 

Public Health 

25 

7.3 

96.8 

13.2 

Health Administration 

9 

10.9 

107.4 

9.9 

Respiratory Therapy 

11 

11.8 

122.0 

10.3 

Gerontology Health Mgt 

15 

13.1 

129.7 

9.9 

Dental Tech. 

22 

11.9 

104.1 

8.8 

Allied Universities 

11 

11.5 

88.1 

7.6 


Table 3. Documents and discussions done by students in the BCC course. 


Semester 

Documents 

Discussion 

2008 

470 

294 

2009 

464 

678 

2010 

631 

764 

2011 

370 

879 

2012 

481 

1257 

2013 

547 

927 


a computer every day just accomplishes its purpose as 
suggested by Malan (2011). Students’ responses indicate 
that these OER videos are interesting and challenging. 
Videos with subtitles are highly proposed by those who 
are not English-native speakers. There are not so many 
courses with full English content in this medical university. 
At present, increasing numbers of international students 
elect this course due to its English-speaking content. 
Students obviously divide into groups of native students 
and international students and the two groups show 
opposite opinion on the language issue. In addition, the 
first version of video was in Flash video format and was 
not mobile-device friendly. These videos have been 
changed into MP4 format in 2014. 

Students have shown the enhancement of self- 
regulated learning through Malan’s OERs. Course 
activities appear to induce students to participate and 
view more. Teaching materials consisting of more 
operations and instructions have gained more views than 
just delivering knowledge content. Although, activities of 
the book club and Jeopardy-like game were not popular 
in this survey, they offered opportunities for students to 
cooperate with each other in the learning process. Few 
students left very encouraging massages in the forum 
and appraised the interactions appear in the online 


competition. During the period of 2008 to 2013, after 
summing up online documents and discussions produced 
by students through LMS records, the BCC course was 
considered to be one of the most “active” courses. The 
amount of each year is indicated in Table 3. 

To manage the course, the instructor used to consume 
considerable amount of time to reply the students’ 
message and setup the LMS content. Thanks to the 
efficient support of the LMS, many logistical issues can 
be resolved by this educational technology. However, to 
calculate peer review scores fairly is still a fairly 
challenging task. A better design LMS function to support 
instructor logistics may be needed. Overall, the BCC 
course in this study has been offered once per year in a 
medical university as a general education course for the 
past 10 years. In the beginning, this course was primarily 
a lecture-based format. Both the instructor and students 
were dissatisfied with either the content or the outcomes. 
After the introduction of innovative course activities, 
students tended to show positive response in course 
satisfaction, even though the new design requires 
considerably more time on assignments and studying. 
Recent surveys show that more than 90% of students 
who took the course reported they have learnt the BCC. 
In addition, more than 90% of students who took the 
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course recommend it to others. These perceptions are 
supported by the fact that the enrollment rate of this 
course has increased in recent years, and it has become 
one of the most popular courses in the university where 
this study was conducted. 

Further research is recommended regarding 
relationships of student majors to outcomes in the BBC 
course. It is expected that students' performance will be 
different among majors in the health science field. 
Traditionally, medical students have higher entrance 
exam scores than other students of health-related majors 
and therefore may be expected to perform better. In this 
course, the result does support this expectation. However, 
further study may be needed regarding these findings 
and the details of relationships between student majors 
and BCC course outcomes. 


Conclusion 

Based on the findings of this study, innovative activity 
design appears to be an effective method to enhance 
computer literacy for undergraduate students with health 
science related majors. Utilization of educational 
technologies, such as LMS and OER, can benefit both 
instructors and students. Further research should be 
undertaken to validate and extend these findings, and to 
examine direct relationships between a computer literacy 
course and student academic performance, both in the 
BCC course itself and subsequently other academic work. 
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